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Abstract 

Wc consider the problem of threshold estimation for autoregressive time scries 
with a "space switching" in the situation, when the regression is nonlinear and the 
innovations have a smooth, possibly non Gaussian, probability density. Assuming 
that the unknown threshold parameter is sampled from a continuous positive prior 
density, we find the asymptotic distribution of the Bayes estimator. As usually in 
the singular estimation problems, the sequence of Bayes estimators is asymptotically 
efficient, attaining the minimax risk lower bound. 
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1 Introduction 

The simplest threshold autoregressive (TAR) process is the time series, generated by the 
recursion 

Xj+i = piXj I{Xj<e} + P2Xj ]I{x,>9} + j = 0, . . . , n - 1, 

where ej ~ !N (O, cr^) are i.i.d. random variables and pi ^ p2 and o"^ are known constants. 
The unknown threshold parameter 6 £ Q = (a, /?) is to be estimated from the data 
X" = {Xq,Xi, . . . ,Xn)- This model and some of its generalizations has been extensively 
studied during the last decades (see e.g. [I]- [5], [9] and the references therein). Particularly, 
much attention focused on the properties of the least squares (LS) estimator 

'^"^ r -|2 

(9; = argmingge Yl [^J+i ~ P^^^ ^{Xj<e} - P2Xj ^{x,>e} ■ 

j=0 

Assuming that V |p2| < 1 and thus that (Xj) is geometric mixing with the unique 
invariant density ip{x,9), Chan [1] proved consistency of 0* with the rate n (rather than 
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-y/n as in regular problems) and showed that the limit distribution is related to certain 
compound Poisson process (see ([2]) below). Note that if ei ~ Tsf (0,(7^), the LS estimator 
coincides with the maximum likelihood (ML) estimator. 

This work continues the study of the Bayes estimator for the TAR models, initiated in 
[3] and developed further in (2] and [1] (see also [7] for the continuous time counterpart). 
We consider the following more general nonlinear TAR(l) model 

Xj+i = h{Xj) l{Xj<e} + giXj) l{x^>gy + Ej+i, j = 0,...,n-l, (1) 

where h{x) and g{x) are known functions, (ej) are i.i.d. random variables with a known 
density function / (x) > 0, x € IR and the initial condition Xq is independent of (sj) and 
has a probability density fo{x). 

Throughout we shall assume that the following conditions are in force 

(al) The parameter G {a, (3) = ©, — oo < q < /3 < oo is sampled from the continuous 
positive prior density p{6) ,9 £ Q. 

(a2) The functions h and g are continuous and satisfy 

inf |5(?;)| > 0, 6 (v) := g (v) - h {v) . 

(a3) The random variables {£j)j>i are i.i.d. with a known continuous bounded density 
function / (x) > 0, x G R 

(a4) The functions h (x) , g (x) and / (x) are such that the time series, generated by ([TJ, 
is geometric mixing with the unique positive bounded invariant density f {x,9), i.e. 
for any measurable function |'0(x)| < 1 



B{i;{Xj)\J,) - / i^{xMx,e)dx 



E 

with positive constants C and r < 1. 
(a5) The function 

f{y + z 



J{z) 
is bounded. 



In- 



f{y) 



f {y) dy, min5(^) < z < max 5(0) 



The likelihood function of the sample X" is given by 

L {6, X") = /o (Xo) n /(^i+i - h (Xj) l{x,<e} - 9 (Xj) l{x,>e}) ' 



3=0 



2 



and the Bayes estimator On with respect to the mean square risk is the conditional expec- 
tation 



B(e\x' 



Lp{e)L{9,x-) de • 



Since the hkehhood L{0,X^) is piecewise constant in 0, the estimate can be computed 
efficiently (see [3]). 

The asymptotic properties of are formulated in terms of the following compound 
Poisson process 



Z{u) 




N+{u) /(f^+W) ' 



u > 0, 
u < 0. 



(2) 



Here is the true value of the parameter, are independent random variables with the 
density function /(x), (•), N- (•) are independent Poisson processes with the same 
intensity A = (^O) ^o) {Z{u) := 1 on the sets {N±{u) = 0}). 
Define the random variable 

fffUZ (n) dn 
Ir Z (u) du ■ 

As shown in [2] (see [6] for the general theory), we have the following lower bound on 
the mean square risk of an arbitrary sequence of estimates {6n)- 

lim lim sup n^Eg (On - 0)^ > Eq^^u^, 

S^On-^oo |6l-6»o|<<5 



and the Bayes estimates (On) are efficient, attaining this lower bound asymptotically. Our 
main result is the following 



Theorem 1.1. Under the conditions (al) (si5), the sequence of estimates [On) is consis- 
tent, the convergence in distribution 



n[0n- Oq 



holds and the moments converge: 

lim n^Efl 



7n — (70 



u 



p > 0. 
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Remark 1.1. The assumption (a4) is often easy to check, using the standard ergodic 
theory as e.g. in f^. The assumption (a5) is satisfied for many common densities. For 
example, for the Gaussian innovations ej ~ N (0,cr^), 

T / N z"^ \z\ 

In this case, the limit compound Poisson process Z (u) has Gaussian jumps: 



In 



f{ef±6{eo))_ 6 (Oof 6(90) 



f{4) 



2a2 



2a^ 



I 1 —1 

Similarly the assumption (a5) is checked for the Laplace density f [y) = {2a) e ~ 
and the limit process has jumps of the form 



In 



f{ef±5i9o)) _ 1 



f{4 



2 The Proof 



We shall verify the conditions of the Theorem 1.10.2 in [6], where the properties of the 
Bayes estimators, announced in Theorem II. H are derived from the convergence of the 
normalized likelihood ratios 



Zn (u) 



L{eo + u/n,X'' 



uG 5J„ = [n(a-0o),n(/3-0o)] 



to the limit process Z {u) , u G K and the two inequalities ^ and (jlOp . presented below. 
The change of variables 9 = 9q + u/n gives 



Then, informally, we have 

^ Iu„uZn{u) du 



l/u„^[p(^o) + o(l)]^"H 
" !^^\pi9o) + o{l)]Zniu)du ■ 



Un = n 



+ o(l) 



IrUZ (u) du 
/r Z (n) du 



u. 



Theorem 1.10.2 in [6] validates this convergence along with the convergence of moments. 
Similar program has been realized in the preceding works [3], [2] and [1]. 
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Remark 2.1. To avoid inessential technicalities, we shall assume that {Xj) is stationary, 
i.e. Xq ~ {p{-,6q). Due to the mixing property (a4j, all the results below can be derived 
without stationarity assumption, along the same lines with minor adjustments (see for 
details). 

Remark 2.2. Below, C , C , c, Cp, etc. denote constants, whose values are not important 
and may change from line to line. We shall denote by Pg and Eg the probability and 
the expectation, corresponding to the particular value of the unknown parameter 9 £ Q 
and set 3'j := cr{ei,i < j}. The standard O(-) and o(-) notations will be used and we set 
ET=ki-)=Oand IYP^,{...) = I for k > m. 

2.1 Convergence of f.d.f. 

We shall prove the convergence of the finite dimensional distributions: 

{In Zniui),..., In Zn{ud)) ^ {In Z{ui),...,\nZ{ud)) , ueR'^, (3) 

following [1]. We shall restrict the consideration to = no < ui < ... < Ud, leaving out 
the similar complementary case. To this end, note that the declared limit process In Z{u) 
has independent increments and 



Egg exp iXj ( In Z{uj) — In Z{uj^i) 



where (recall that 6 := g — h) 



^(A,):=E,„exp (^lA, In :^fci±^) . 

Since InZ(O) = a.s., ([3]) follows from the convergence of characteristic functions of the 
increments 

limEg^ exp lAj (lnZ„(nj) - lnZ„(nj_i)) j = e^(^), A G R'^. 

Let m{x,9) := h{x)TL^^^gj + g{x)'K^^>0j and note that 

m{x, 6q + Uj^i/n) — m{x, 6q + uj/n) = 
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where := 



9o + Uj-i/n,6o + Uj/n). Let [B"_^ := [9q,9q + Uj^i/n), then 



In Zn{uj)— In Zn{uj^i) = y In 



k=o f(^Xk+i-m{Xk,9o + Uj^i/n)^ 
^ f(ek+i+m{Xk,eo)-m{Xk,eo + Uj/n) 
k=o f[£k+i +m{Xk,Oo) -m{Xk,9o + Uj-i/n)j 



^ fUk+i + S{Xk)l{x,eB- J + 6{Xk)lix,eu- 
y ^ In — -: 

k=o /(sfc+i + '^(^fc)I{XfeeB"_J 



(4) 



n-l 



k=0 
n-l 

k=0 



{XfceD^"}ln 



fUk+l + (^(Xfc)]I{Xfe6B"_J + (^(^fe! 



A;=0 



/(efc+i) 



where the equahty f holds Peg-a.s., since Pe^yXk-i G 

d 

Sn ■■= 'y^Xji^lnZniuj) -\nZn{uj-i) 



0. Further, define 



d n-l 
j=l k=0 



We shall partition n terms of this sum into n^/^ consecutive blocks of size n^/^ and discard 
from each block its n^/^ first entries. As we shall see, this does not alter the asymptotic 
distribution of 5„, but makes the blocks almost independent. Since in each block, the single 
event {Xk G -D"} occurs with probability of order n^/^, the Poisson behavior emerges. To 
implement these heuristics, define 



,1/2 



m = 1, n 



1/2 



J=l fc=(m-l)ni/2+„i/4 



and set Sn '■= Ylm=i ^rn,n (this is the sum, in which the v}^^ entries of each block have 
been discarded). By the triangle inequality 



< 



J-Sn 



+ 



,1/2 



+ 



i5i. 



,1/2 



(5) 
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We shall show that all the terms on the right hand side vanish as n — t- oo. By stationarity 



and the assumption (a5 



< E, 



eo 



gi5„ _ gi5, 



n"' 'max|Aj|E0„I|XoeD"} 



^3/4 
^3/4 
3/4 



In 



<E9o 
f{ei + 6{Xo)) 



< 



max\Xj\Eeol{XoeDnJ{^(.^o)) < 



n 

n"^/^ max I Aj I ^^^^^ — sup v3(x, 9q) sup J {6(9)) 



1 ' n xGR 

i.e. the first term in ([5]) converges to zero. 



^0, 



Further, note that by the Markov property of (Xj) and (a4) 



'"(£_l)„l/2 j - EggC 



pi's!.!! 



1/4 



£= l,...,n 



1/2 



and hence 

Ee^e'^" - (Ee„e 



£=1 ^ m=] 

E n 
E ( n 



nl/2 






ei^™. _ ( 


m=l 








N^ni/2_£ 


m=l 



,1/2 



„l/2„^N 



Efl„ e 



i5p 



m=l 



£=1 



Eeo(e'^^-|J, 



(^-l)ni/2 



(£_l)„l/2 

Eeoe^^^ 



Eeoe'^^-")(Eeoe'^^' 



oi'S'l,: 



il/2_£ 



,1/4 n— >-oo 



> 0. 



< 



It is left to show that the last term in ([5]) converges to zero. Let D" = U^=i ^^'^ 
introduce the following events 

d ni/2 

^0 := n ^ ^1 •= U U ^^.i' ^2+ := (^0 U Ai 

£<„l/2 j = l f=0 

:= {Xk G D^"} n fl {Xe^ D*^}. 
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In words, Aq is the event, on which none of the first n^/^ samples falls in any of D"'s, Ai 
is the event of having exactly single sample visiting one of D"'s, etc. On the event Akj, 



i=i ^=„i/4 J[^e+i) /(efc+ij 

and, since {Xk G D^^} = A^j l±| ({X,. G D«} n [j.^ki^i G D"}) , 

d ni/2 d nV4_i 

j=l k=0 j=l k=0 

2^ 2^ ^00 exp l^iAj In J [Hx^eD]} " I{x,,eDy}nU,^ jx.eo"} ) • 

By continuity of (p{x,6q) and 5(x), 



and 



Eso exp I lAj In , ^ 1 2{x,eDy} 



/(efc+i) 

Eg, exp (^lAj In J(^^^ ) ^x^eD^} + o{n ) 



Further, by the Markov property, for k < i 

Peo{Xk G D^,Xi G D") = Ee,lx,eD-Peo{Xe G D"|3-,_i) = 

where the inequalities hold, since the density /(x) and therefore the invariant density 
ip{x,9o), X G [R are bounded. Similar bound holds for > £ and it follows that 



P^o ({Xk G D-} n U {X, G D"}") < E (^fc e X, G D") < C^^n-^l^ 
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Plugging these estimates into ([6]), we get 

d 



If we set all A/s to zeros, we also obtain 



Further, 



KnM"^ j=l 



^ ^'"E (^J ~^J-0'^(^o,6'o) + o(n 



On the other hand, P£)q(^o) ^ 1 ~ P6»o(^i) ^^"^ i'^ view of ([7]), it follows that 

d 

(^0) = 1 - ^ (u^. _ u,.^)^{Q^M + o(n-i/2). 

Finally, using ([7]) and dH), we also have 

Pe„(A2+) = 1 - P^^o(^o) - Peo(^i) = 
Assembling all parts together, we obtain the asymptotic 



n 



and, in turn. 



lim 

n 



(E,,e'^^-) -e 



//(A) 



The claim now follows from 



2.2 Equicontinuity 

The next step is to show that for some C > 



(^Z^/^ (ns) - Z'^' (ui)j <C\U2- nil . 
As in Lemma 2.4 in [3], for e.g. n2 > ui > 0, (jH) gives 



(9) 



-'eo+«i/n 



In 



Zn (^l) 
Zn (U2) 



n-1 



< E 



hi. 



/(ej+i) 



^{eo+^<X,<0o+^} 



In 



/(^i+i) 



+Mi/n 

i=o 

n-l 

i=o 

n-1 

= n J {6 (x)) 99 (x, 00 + ui/n) dx < C \u2 — ui\ , 

J 00+^ 

as required. 

2.3 Large deviations estimate 

FinaUy we shall prove that for any p > there exists a constant Cp > such that 

Cn 



(10) 



We shall only sketch the proof, as most of the arguments can be directly adopted from 
the proof of Lemma 2.2 in or Lemma 2.5, [4]. Note that for any c > 0, 



{zi/''(u)>e-'=\^\} 



+E,„zy2(„)i 



< 



{Ee,Zr.iu)f'p'j;{zl/\u) > e-l"l) +e-^l"l =Pjf (lnZy2(„) > +e-<^H (11) 

and hence it suffices to show that for some c > 0, 



Cn 



Peo{lT^Zl/^{u)>-c\u\) <-^, p>0. 
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For n > (and similarly for n < 0), 
Peo {in^y (u) > -c|n|} 



n-l 



j=0 



f{6{Xj)+ej+i) 



Let £ (x, y) := In 



fiy) 



f (ej+i) 
and introduce the notations 



G{6) = -lnH{6) 



n-l 



n-l 



j=0 j=0 



where B*^ = [^0)^0 + ^^/'^] and 



f (y) dy 



is the Hellinger integral of order 1/2. By the Jensen inequality for all 5^0, H{5) < 1 
and hence G (5) > 0. 

Further, we have the following identity 



Eeoca*" = 1. 



(13) 



Indeed 



3" n- 



and 



T 

^ n- 



3'n-l j I{X„_ieB"} + I{X„_i0B"} 



/ /((^(x„„i) + jy)V/^ 

I fiy) 



j / (y) dy I{x„_i6B"} + ]I{x„_i^B"} 



exp ( - G {5 (X„_i)) I|x„_igB"} 
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Hence 



and (1131) follows. Now we have 



n-1 



do 



j=0 



where we used (I13p . In view of (llip and (|12p . it is left to show that for all p > 1, 

' n-l 



n\ < -CU > < 



Cn 



j=0 



\u 



p • 



Following [3], we shall split the consideration into the cases u < n'^ and < u < 
n{j3 — a), for some s € (0, 1). 

To this end, note that the Hellinger integral H[5) is a continuous function of 6: 



{HiS)-HiS + r,)y 



\ fiy) 



f{y) 



) f{y)dyj < 



(f{5 + y)V'^ ff{s + v + y) 



\ fiy) 



f{y) 



l/2^ 



f{y)dy 



yW+yW+v + y)dy = / (^/m - Vfiv + y)) dy < 



|/(y)-/(r? + y)My ^0 



where we used LeCam's inequality for the Hellinger and the total variation distances and 
the convergence holds by Scheffe's lemma. 

By continuity of G(b) = — lnff(5) and since G{5) > for all 5 7^ 0, the assumption 



(a2) implies that for u < 

G{5{X^))l{x,^^^}> inf G(<^(;;))>co 



with some constant cq > and 



|5(2)<^cu}<p,„|x;2 



3 c 
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Now let r]j (u) = Eq^^TL^x^gb"} ~ ^{XjGb"}- Since the density f{x) is continuous and 
positive, so is the invariant density (p {x,6q) and 



n-l 
j=0 

with a positive constant C . Then 

3 c 



n 



do+u/n 



if (x, ^o) dx > C' u, 



Peo< 



n-l 



2 CO 



■ u 



71 — 1 ^ 



3 c 

1 

2 CO 



n-l 



j=0 



E 



En— 1 
j=0 1j 



u 



2p 



KU 



2p 



where we chose c small enough, so that C" — §c/co = k > 0. Using the geometric mixing 
property (a4) and an appropriate version of Rosenthal's inequality as in Lemma 2.2 [3], 
we get 



2p 



which yields (llOp for |u| < n^. The complementary case, < \u\ < (/? — a)n is treated 
exactly as in Lemma 2.2, [3] or Lemma 2.5, [1]. 



3 Discussion 

Theorem 11.11 can be directly generalized to the multi-threshold autoregression 

K 

^j+i = X/ ^-^^^ ^{dk<Xj<ek+i} + ^i+i) j = 0, 1, . . . ,n, 
fc=o 

where 9 = (6*1, ... , 0/^) is the unknown parameter (and 6*0 = — oo and 9k+i = oo are set). 
As in ([1]), (ej) are independent random variables with known density / (x) > 0, x E R and 
the functions /jfc (") and / (•) are continuous and such that (Xj) is geometrically mixing. 
Assume that 9k G {ak,h)-, where fit < Oik+i- 
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For all sufficiently large n and Uk > 0, the normalized likelihood ratio is given by 



(Uj — J_J_ J — 

j=o f [Xj+i - J2k=o (Xj) 2{0fc<x,<efe+i} 



Yi^f{j:Lohk{x,) 



,=0 ^(^^+1) 



and thus 



lnZ,(u)=Vln^ ^-^ 

2^ 2^ 1^ 777—) \e,<x,<e,+--^y 

k=l j=o V J + v 

where 6k{x) := hk-i{x) — hk{x). Using the same approach as in the proof of Theorem 11.1 
it can be seen that 

, , , , f f{<i + ^k{ek)) 

lnZ„(u) =^ 



fc=i 1=1 



where Nj^ (ufc) , > are independent Poisson processes with intensities (p{9k,6k) and 
e'^i are i.i.d. random variables with the density /. Similar asymptotic is obtained for 
ttfc < 0. Consequently the limit likelihood ratio is a product on K independent one- 
dimensional copies of the process defined ([2]) (with ^0 replaced by 9k s) and the cor- 
responding Bayes estimates 9k,n, k = 1,...,K are asymptotically independent with the 
asymptotic distribution as in Theorem ll.il 
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